1. From Raw Output to Reportable Results

1.1 What a simulation actually produces

• Time series — state sampled every step or every event (queue length, temperature, population).
• Event traces / logs — one record per event; the raw material for verification and for post-hoc statistics.
• Summary statistics per replication — the mean waiting time of run 7.
• Fields / snapshots — the whole mesh or grid at selected times; large.
• Scenario matrices — one row per parameter combination in a sweep.

1.2 The processing pipeline

1. Collect in a machine-readable format — CSV or Parquet for tables, HDF5/NetCDF for fields, JSON for metadata. Never in a format that only your GUI can read.
2. Clean and truncate — discard the warm-up period (Unit IV), handle incomplete runs explicitly rather than silently.
3. Aggregate — means, quantiles, time-weighted averages, and confidence intervals across replications.
4. Compare — differences between scenarios with intervals, not bare numbers.
5. Present — the subject of Sections 2–4.
6. Archive — results with the model version, parameters, seeds and software versions, so the figure can be regenerated a year later.

2. Display Forms I: Tables and Graphs

2.1 Tables

A table is the right display when the reader needs exact values, when there are few numbers, when the quantities have different units, or when the result must be looked up rather than compared visually.

1. One row per scenario, one column per measure; keep the ordering meaningful (by performance, not alphabetical).
2. Report estimate ± half-width, or the interval, with the confidence level and the number of replications stated in the caption.
3. Round to the precision the confidence interval justifies. Writing 4.283719 minutes when the interval is ±0.4 is false precision and loses marks in a report.
4. Right-align numbers, use a consistent number of decimals, and state units in the header.
5. Highlight the best value per column, but never colour a table so heavily that it becomes a bad graph.

2.2 Graphs: choosing the right chart

2.3 Principles of honest graphics

1. Show the uncertainty. A mean without an interval is an opinion. Error bars, bands, or all replication traces in light grey behind the mean.
2. Do not truncate a bar-chart axis at a non-zero value — it exaggerates differences. Line charts of a narrow range may be zoomed, but say so.
3. Maximise the data-ink ratio (Tufte): delete gridlines, 3-D effects, shadows, and decorative fills. A 3-D pie chart of simulation results is indefensible.
4. Label directly where possible instead of forcing the reader into a legend.
5. Use colour meaningfully — sequential palettes for magnitude, diverging for deviation about a reference, categorical for unordered classes. Avoid rainbow scales: they create false boundaries and are not perceptually uniform. Prefer viridis or cividis.
6. Design for accessibility — about 8% of male readers have a colour vision deficiency; encode with shape or line style as well as colour, and check the figure in greyscale.
7. Keep the same axes when comparing panels, otherwise the comparison is visual nonsense.
8. Caption completely: what is plotted, how many replications, what the band means, and which model version produced it.

3. Display Forms II: Multidimensional Visualization

Simulation output is rarely two-dimensional: a parameter sweep may have six inputs and four outputs, and a mesh simulation has three space dimensions plus time plus several fields. Multidimensional visualization is the set of techniques for showing more variables than a plane naturally allows.

3.1 Visual channels for extra dimensions

Beyond x and y, a plot can encode variables in colour, size, shape, orientation, texture, opacity, animation over time, and position in small multiples. Ranked by accuracy of human perception (Cleveland & McGill): position > length > angle > area > colour saturation. Put your most important variable on position, never on colour intensity.

3.2 Techniques for many variables

3.3 Scientific (field) visualization

For mesh-based and particle simulations the data is a field over space and time, and a different toolkit applies:

• Slices and cutting planes — reduce 3-D to 2-D at a chosen location.
• Iso-surfaces and contours — the surface where a field equals a chosen value (e.g. a temperature front).
• Volume rendering — map scalar value to colour and opacity through a transfer function; shows interior structure but is sensitive to that function.
• Vector-field methods — arrow glyphs, streamlines, pathlines, streaklines, line integral convolution.
• Feature extraction — detect and track vortices, shocks, cracks or clusters, then visualise only those; essential when the raw field is too large to store.
• Animation — time as time. Powerful for communicating dynamics, but useless for precise comparison, so always pair an animation with static summary charts.

Tools: ParaView/Catalyst, VisIt/Libsim, Ascent for in-situ and large-field visualization; Matplotlib, Seaborn, Plotly, Bokeh, Altair and D3.js for statistical and web graphics; Tableau and Power BI for dashboards.

4. Interfaces: Terminals, X and MS Windows, and the Web

The syllabus lists these three interface generations because each is still in use, and each is the right answer in different circumstances.

4.1 Terminal / command-line interfaces

• What it is: the simulation is run as a program with arguments or a configuration file, writing text and result files; possibly with a curses/TUI progress display.
• Why it still dominates HPC: it is scriptable, remote-friendly over SSH, works under batch schedulers (Slurm, PBS) where no display exists, composes with shell pipelines, and is trivially reproducible — the command line is the provenance record.
• Design rules: every parameter settable from a config file; a --seed option; deterministic output paths; progress and errors to stderr, data to stdout or a file; a non-zero exit code on failure so a workflow can detect it.
• Limits: no interactive exploration, steep learning curve for domain experts who are not programmers.

4.2 X Window System and native desktop GUIs

• X11 is the classical UNIX windowing system, built on a client–server, network-transparent model: the X server owns the display and input devices on the user's machine, while the application (the X client) may run on a remote compute server and draw across the network. This is why ssh -X can display a simulation's window from a cluster on your laptop — the property that made X the standard for scientific workstations. Toolkits: Motif and Xt historically; Qt and GTK today; Wayland is the modern replacement for X on Linux desktops.
• MS Windows GUIs use an event-driven message-loop model (WinAPI, MFC, .NET WinForms/WPF), and commercial simulation packages (Arena, Simul8, FlexSim, MATLAB) established the familiar pattern: a model canvas, a properties panel, run controls, animation, and a report window.
• Strengths: rich interactivity, animation of the running model (which is a superb verification tool — you literally watch entities move and immediately see logic errors), direct manipulation of the model, immediate feedback.
• Limits: platform-bound, installation and licensing overhead, click-driven workflows that are hard to reproduce or to batch, and poor performance over a slow network link.

4.3 Web interfaces

The modern default for delivering simulation to users, and the target of your laboratory Experiment 10.

1. Architecture: a browser front end (HTML/CSS/JavaScript, or a framework such as React) talks over HTTP/REST or WebSocket to a back end (Flask/FastAPI/Django, Node) that owns the simulation engine and a job queue; results are stored and returned as JSON or files.
2. Patterns: submit-and-poll for long runs; WebSocket streaming for live progress and animation; server-side rendering of large images (the in-situ idea again) versus client-side rendering with Plotly, D3 or WebGL/three.js for interactive small data.
3. Dashboard frameworks that avoid writing JavaScript: Dash, Streamlit, Panel, Bokeh server, Shiny, Voilà — the pragmatic choice for research tools.
4. Strengths: zero installation, cross-platform, shareable by URL, naturally multi-user and collaborative, easy to place in front of a cluster, and the same interface serves a laptop and a phone.
5. Challenges: long-running jobs do not fit the request/response model (needs a queue and job IDs); large result transfers; authentication, authorisation and quota control; reproducibility of user-driven sessions; and browser resource limits for very large geometry.
6. Security duties when a web page can launch computation: validate and bound every parameter server-side, never pass user input to a shell, rate-limit and quota jobs, run the engine in a sandbox or container with resource limits, and authenticate before allowing expensive runs.

The mature answer is that these are layers, not rivals: a scriptable engine at the bottom, a documented API in the middle, and any number of interfaces (CLI, notebook, desktop, web) on top. Never bury the simulation logic inside the GUI — that is the design error that makes a tool impossible to batch, to test or to reuse.

5. Validation of Model Results

5.1 Verification techniques

1. Structured walkthrough of the code and the event logic with a colleague.
2. Trace analysis — follow a few entities event by event and check by hand that state changes are correct.
3. Degeneracy and extreme-condition tests — zero arrivals, one entity, infinite service rate, zero capacity. The output must be obviously right (or the model must correctly refuse).
4. Comparison with analytical results — simulate M/M/1 and check L and W against the Unit IV formulae; this is the single most valuable verification test for a queueing simulator.
5. Consistency checks and invariants — conservation of entities, mass or energy; Little's law on the measured output; non-negative queues.
6. Continuity and monotonicity tests — a small change of input should give a small change of output unless a threshold is crossed; more servers must not increase waiting.
7. Seed and independence checks — the same seed reproduces the run exactly; different seeds give different but statistically consistent runs.
8. Convergence / mesh refinement for continuous models (Unit I).
9. Unit and regression tests in the software-engineering sense, run automatically.

5.2 Validation techniques

Naylor and Finger's three-step approach is the classical framing, and Sargent's catalogue is the standard list of techniques; both are worth citing.

(a) Face validity and conceptual validation

• Face validity — do domain experts, shown the model's behaviour, judge it reasonable? Subjective but the cheapest way to catch a structurally wrong model.
• Assumption review — every simplification listed and justified against the intended purpose.
• Theoretical validity — the mechanisms agree with accepted theory in the domain.

(b) Input data validation

Garbage in, garbage out: distributions must be fitted and tested (Unit IV), data sources documented, and the period covered stated. A model can be perfectly implemented and still wrong because the arrival rate came from an unrepresentative week.

(c) Output (operational) validation — the core

1. Comparison with historical/field data. Run the model under the historical conditions and compare output distributions with observed ones.
2. Statistical comparison. Use a confidence interval on the difference between simulated and observed means; two-sample t-tests; Kolmogorov–Smirnov or Anderson–Darling to compare whole distributions; time-series comparison via spectral or autocorrelation analysis. Report error metrics:

3. Data splitting. Calibrate on one part of the data, validate on a held-out part. Fitting and validating on the same data proves nothing.
4. Predictive validation. The strongest evidence: the model predicts an outcome that is then observed.
5. Event validity. Compare the number and timing of characteristic events (failures, peaks, blockages), not only the means.
6. Turing test. Show experts mixed real and simulated output records and ask them to identify which is which. If they cannot do better than chance, that is meaningful evidence.
7. Extreme condition and stress validation. The model must behave plausibly under conditions outside normal operation.
8. Sensitivity analysis as validation (Unit IV) — the model's response to each factor should agree with the direction and rough magnitude that domain experts expect.
9. Docking / cross-model validation. Compare against an independent model of the same system, ideally built by a different team or in a different paradigm.

5.3 When you cannot validate

Some models have no data to validate against: a system not yet built, a pandemic that has not happened, a far-future scenario. Then you must be explicit about what you can establish:

• Verification of the implementation, in full.
• Validation of the components and sub-models against data where it exists.
• Structural/theoretical validity of the mechanisms.
• Behavioural validity: the model reproduces qualitative patterns known to occur (pattern-oriented modelling).
• Thorough sensitivity and uncertainty analysis, reported as ranges rather than point predictions.
• An explicit statement that the results are conditional projections under stated assumptions, not predictions. This is an ethical requirement (Unit I, Section 11), not merely a stylistic one.

5.4 Reporting: the credibility checklist

1. Purpose and the questions the model was built to answer.
2. Conceptual model, assumptions and their justification.
3. Input data sources, fitted distributions and goodness-of-fit evidence.
4. Verification evidence (tests performed and passed).
5. Validation evidence with error metrics and the comparison data.
6. Experimental design: warm-up, run length, replications, seeds.
7. Results with confidence intervals and the appropriate display forms.
8. Sensitivity and uncertainty analysis.
9. Stated domain of validity and known limitations.
10. Provenance: model version, code repository, parameters and software environment.

6. Unit Summary

• Raw output becomes a result through a pipeline: collect in open formats, truncate the warm-up, aggregate with intervals, compare, present, archive with provenance.
• Tables give exact values with intervals and honest precision; charts answer specific questions — evolution, distribution, comparison, relationship, spatial structure.
• Honest graphics: show uncertainty, do not truncate bar axes, maximise data-ink, label directly, use perceptually uniform colour maps, design for colour-vision deficiency, caption completely.
• Multidimensional visualization adds channels (colour, size, shape, animation, small multiples) and techniques (SPLOM, parallel coordinates, heatmaps, dimensionality reduction, linked brushing).
• Field data uses slices, iso-surfaces, volume rendering, streamlines and feature extraction; in-situ visualization avoids the I/O wall.
• Interfaces are layers: a scriptable engine (CLI, reproducible, batch), native GUIs (X11's network-transparent client–server model; Windows message loops; animation for verification), and web front ends (zero install, shareable, but needing job queues and security).
• Verification asks whether the model was built right; validation asks whether the right model was built; accreditation is the formal acceptance for a stated use.
• Validation combines conceptual, input-data and operational layers, uses held-out data and error metrics, and must state the domain of validity.
• Where validation data does not exist, report conditional projections with explicit assumptions and uncertainty — never predictions.

6.1 Key terms

Provenance · warm-up truncation · data-ink ratio · small multiples · parallel coordinates · SPLOM · linked brushing · perceptually uniform colour map · iso-surface · volume rendering · streamline · in-situ visualization · X client–server model · network transparency · message loop · REST / WebSocket · submit-and-poll · verification · validation · accreditation · face validity · event validity · docking · Turing test · RMSE · MAPE · held-out data · domain of validity.

6.2 Practice questions

Short answer (2–3 marks each)

1. When should a table be used instead of a graph?
2. Why should a bar-chart axis not be truncated?
3. What is the data-ink ratio?
4. What is in-situ visualization and why is it needed?
5. Explain the client–server model of the X Window System.
6. Distinguish verification, validation and accreditation.
7. Define face validity and event validity.

Medium answer (5 marks each)

1. Describe five chart types and the question each answers, with a simulation example for each.
2. Describe three techniques for visualising more than three dimensions, with their limitations.
3. Compare terminal, desktop GUI and web interfaces for a simulation tool under at least four criteria.
4. List and explain six verification techniques for a discrete-event simulation.
5. How would you validate a model when no historical data exists?

Long answer (10 marks each)

1. Discuss display forms for simulation results — tables, graphs and multidimensional visualization — including selection criteria, principles of honest presentation and tools.
2. Explain the validation of simulation results in full: verification versus validation, conceptual, input and operational validation, statistical comparison techniques, error metrics, and the credibility checklist. Illustrate with one worked example.
3. Design the complete result-viewing subsystem for a simulation tool: data formats, analysis pipeline, display forms, and a web interface architecture with its security and reproducibility requirements.
4. “A simulation result without uncertainty and provenance is not a result.” Discuss, with reference to output analysis, presentation practice and professional ethics.

6.3 Further reading

• R. G. Sargent, “Verification and validation of simulation models”, Winter Simulation Conference — the standard reference for Section 5, updated most years.
• A. M. Law, Simulation Modeling and Analysis — chapters on verification, validation and output presentation.
• E. R. Tufte, The Visual Display of Quantitative Information — for Sections 2 and 3.
• C. Ware, Information Visualization: Perception for Design.
• J. Banks (ed.), Handbook of Simulation — chapters on output analysis and model credibility.